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— The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1,136 (a). In no event, however, may a reply be timely filed after SIX (6) MONTHS from the 
mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply wrthin the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if tinnely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

Responsive to communication(s) filed on Apr 2 1, 2003 



2a) K This action is FINAL. 2b) □ This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Exparfe Quayle, 1935 CD. 11; 453 O.G. 213. 
Disposition of Claims 

4) K Claim(s) h 2, 4, and 6-15 



4a) Of the above, clalm(s) 
5)0 Claim(s) 



6) K Claim (s) 2, 4, and 6-15 

7) 0 Claim (s)_ 

SjD Claims 



is/are pending in the application, 
is/are withdrawn from consideration. 

is/are allowed. 

is/are rejected. 

is/are objected to. 



are subject to restriction and/or election requirement. 



Application Papers 

9)0 The specification is objected to by the Examiner. 



10)0 The drawing(s) filed on 



is/are a) □ accepted or b)n objected to by the Examiner. 



Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85{a). 

1 DD The proposed drawing correction filed on is: 3)0 approved b)^ disapproved by the Examiner, 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) 0 Acknowledgement is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)n All b)Q Some* c)^ None of : 

1 . □ Certified copies of the priority documents have been received. 

2. □ Certified copies of the priority documents have been received in Application No. . 



3. □ Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
*See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgement is made of a claim for domestic priority under 35 U.S.C. § 1 19(e). 
a)n The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgement is made of a claim for domestic priority under 35 U.S.C. §§120 and/or 121. 

Attachment{s) 

1) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(8). 

2) Q Notice of Draftsperson's Patent Drawing Review {PTO-9481 5) Q Notice of Informal Patent Application (PTO-152) 

3) Q Information Disclosure Statement(s) (PTO-1449) Paper No(s). 6) Q Other: 
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DETAILED ACTION 

1 . This Office Action is in response to the AMENDMENT entered April 2 1 , 2003 for the 
patent appHcation (09/488,028). 



2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed publication 
in this or a foreign country, before the invention thereof by the applicant for a patent. 

3. Claims 1, 2, 4, 6, 7, 12-15 are rejected under 35 U.S.C. 102(a) as being anticipated 
by Kahn et al., "Real-time Gesture Recognition with the Perseus System." 

The present invention is directed to video camera targeting systems that locate and acquire 
targets. Similarly, Kahn et al (Kahn) is also directed to a video camera targeting systems. 
Consequently, the claimed subject matter is also disclosed by Kahn. 

Regarding claim 1, Kahn discloses a method of locating an image. The method includes 
recognizing when and where people point to objects and identifying and locating the object 
pointed to (see section 2 , the Pointing Task). Once the object is located (via locate method), 
certain description and properties/parameters (or characteristics) information of the region 
(person and target location) are measured and stored in the Object Representations (OR) (see 3 J 
Object Representations), The locate method then places a marker on the region parameterized 



Claim Rejections - 35 USC § 102 
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with a tacking function. In the end the camera (Fig. 1) is directed to the sensed object. Moreover, 
The Perseus system (see Fig .1) is capable of sensing a triggering event, such as a pointing gesture 
by a human operator (see section 3.7 Figs); during a pointing task the Perseus system uses, 
receives or access object description information or Object representation (OR) of the person and 
the item being pointed to. In response to said sensing and receiving the Perseus system then 
locates (via camera) the person, waits for the person to point, and retums either a message 
describing which item was pointed to or a message saying nothing was found (see sections 3.6-4). 

Claim 12 recites several similar feature as claim 1 except for "scanning and area..." and 
"storing information...". Kahn further discloses these limitations. Kahn discloses the area the 
finger points is examined (or scarmed), such as through camera, and if an object is found it is 
identified (see The Pointing Task section, and Fig. 6). As mentioned in the rejection of claim 1 
above, storing the properties/parameters information of the region (target location) are measured 
and stored in the Object Representations (OR) (see 3.3 Object Representations section). 

Claim 13 recites a method of aiming at a target. Kahn discloses inputting a pointing 
gesture indicating a direction of the target. Inputting further information of the target such as the 
properties/parameters (or characteristics) information of said region (target location). Directing a 
camera at said target using the properties/parameters information of said region (target location) 
and/or natural language description command (also see 3.6 Long Term Visual memory section). 

Claim 14 recites a method of acquiring a target. Kahn, as described in the above claims, 
discloses inputting through sensing a gesture indicating a direction of said target (Fig. 6). Then, 
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directing (or orienting) the camera (Fig. 1) with respect to said target (see 3.7 Visual Routines 
section). 

Regarding Claims 2 and 7, the features of these claims are discussed in the rejection of the 
above claims. 

Claim 4 calls for receiving speech from said human operator. Kahn teaches this limitation 
(see 3.6, Long Term Visual Memory). 

Claim 6 calls for speech and machine sensor assisting in locating said target. Kahn also 
teaching that in addition to sensing a gesture indicating a direction, he also suggest the use of 
speech in collaboration with gesture in assisting locating a target (see sections 3.6-3.7.1). 

Claim 15 calls for orienting a camera (see the rejection given to claim 14). 



4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 

rejections set forth in this Office action: 

fa) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

§. Claims 8-11 are rejected under 35 U.S.C. 103(a) as being unpatentable over Kahn et 

ah, "Real-time Gesture Recognition with the Perseus System" in view of Mark Billinghurst 



Claim Rejections - 35 USC § 103 



'Tut That Where? 
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While Kahn's "Real-time Gesture Recognition with the Perseus System" storing target 
object representations and Kahn further describes Object Representations (OR) are not static 
representations, the data they contain changes over time (see section 3.3). But Kahn fail to 
suggest that object representations including natural language description representations or voice 
input are stored in a table, look-up table format or in claim language "processing said voice 
information through a look-up table corresponding said speech to search criteria for use with at 
said least one sensor" is not shown. However, Billinghurst 's Put That Where?" stores speech 
information in parallel with gesture information in a look-up a table for later processing (such as 
searching) (table 1). Therefore, it would have been obvious to one having ordinary skill in the art 
at the time of the invention was made to incorporate the speech modality interaction stored in a 
table format of Billinghurst v^th gesture interaction of Kahn because the use of speech 
interaction in association with gesture modality will enhance Kahn's system. Furthermore as 
suggested by Billinghurst (page 61) hand gesturing and speech complement each other, and a 
spoken vocabulary has a more standard interpretation than gesture. 



6. Applicant's arguments filed April 21, 2003 have been fully considered but they are not 
persuasive. 

The Applicant argues that Kahn does not disclose the limitations of claim 1 . hi contrast to 
the argument Kahn does disclose the limitations of claim 1. As described in the rejection of claim 



Response to Arguments 
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1, the Perseus system (see Fig .1) is capable of sensing a triggering event, such as a pointing 
gesture by a human operator (see section 3.7 Figs); during a pointing task the Perseus system 
uses, receives or access object description information or Object representation (OR) of the 
person and the item being pointed to. In response to said sensing and receiving the Perseus system 
then locates (via camera) the person, waits for the person to point, and retums either a message 
describing which item was pointed to or a message saying nothing was found (see sections 3.6-4). 

Furthermore, AppUcant states that Perseus registers an object prior to being able to 
identify the object via the gesture. In contrast to the applicant statement, it is not required to 
register of an object prior to being able to identify the object via the gesture. As described in the 
Introduction section, "Higher level information allows Perseus to select the best visual operators 
for interpreting the scene in a given situation. Once the visual operators are chosen the higher 
level knowledge may also be used to time them to maximize their performance." Thus, Perseus 
architecture takes advantage of the higher level information stored (or registered) in LTVM 
(knowledge about the environment), not as a requirement but as to better understand the pointing 
gesture and as to maximize the performance. 

Conclusion 

% THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS fi*om the mailing date of this action. In the event a first reply is filed within TWO 
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MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 
1 .136(a) will be calculated fi-om the mailing date of the advisory action. In no event, however, 
will the statutory period for reply expire later than SIX MONTHS from the mailing date of this 
final action. 

%, Any inquiry concerning this communication or earlier communications from the Examiner 
should be directed to Tadesse Hailu, whose telephone number is (703) 306-2799. The Examiner 
can normally be reached on M-F from 10:00 - 8:30 ET. If attempts to reach the Examiner by 
telephone are unsuccessfiil, the Examiner's supervisor, John Cabeca, can be reached at (703) SOS- 
S1 1 6 Art Unit 21 7S CPK 2-4A51 . 

9. Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the Group receptionist whose telephone number is (703) 305-3900. 
Ia4(e44e ^^W^ June 1 1 , 2003 




TECHNOLOGY CENTER 210; 



